A potential mechanism for low tolerance feedback loops in social media flagging systems

Many people use social media as a primary information source, but their questionable reliability has pushed platforms to contain misinformation via crowdsourced flagging systems. Such systems, however, assume that users are impartial arbiters of truth. This assumption might be unwarranted, as users might be influenced by their own political biases and tolerance for opposing points of view, besides considering the truth value of a news item. In this paper we simulate a scenario in which users on one side of the polarity spectrum have different tolerance levels for the opinions of the other side. We create a model based on some assumptions about online news consumption, including echo chambers, selective exposure, and confirmation bias. A consequence of such a model is that news sources on the opposite side of the intolerant users attract more flags. We extend the base model in two ways: (i) by allowing news sources to find the path of least resistance that leads to a minimization of backlash, and (ii) by allowing users to change their tolerance level in response to a perceived lower tolerance from users on the other side of the spectrum. With these extensions, in the model we see that intolerance is attractive: news sources are nudged to move their polarity to the side of the intolerant users. Such a model does not support high-tolerance regimes: these regimes are out of equilibrium and will converge towards empirically-supported low-tolerance states under the assumption of partisan but rational users.

1 Sensitivity Analysis 1 In this section we test how robust our results are. We start by analyzing the effect of the 2 models' parameters and then we move onto the effect of the models' starting conditions. 3 4 In the main paper, we base our results on the Relative model with δ = 0.9. First, we 5 verify what happens when we vary δ. We remind that δ regulates how distant φ l is from 6 φ r , i.e. φ l = δφ r . 7 Figure S1 shows the distributions of flags for different φ r and δ values. It is a 8 reproduction of Figure 4 in the main paper. We only show the Kernel Density 9 Estimations for clarity. We can see that we confirm the main result of the paper: in the 10 Relative model for φ r ≥ 0.3 there are asymmetric flag peak probabilities, with the right 11 side of polarity attracting more flags. 12 As δ shrinks, the difference between φ r and φ l grows. The effect is that the 13 left-leaning news sources gets flagged less and less, while the flagging peak for 14 right-leaning sources moves toward zero. We interpret this result later in this section, as 15 it requires more information to be properly understood. 16 We now turn to considering an alternative model: the Subtraction model. In the 17 Subtraction model, φ l = φ r − δ. Just like before, we test different values of δ.

Parameter Sensitivity
18 Figure S2 shows the distributions of flags for different φ r and δ values -again only 19 showing the Kernel Density Estimations for clarity. Its interpretation is the same as 20 Figure S1. Also in this case, we can confirm the main result of the paper: in the 21 Subtraction model for φ r ≥ 0.4 there are asymmetric flag peak probabilities, with the 22 right side of polarity attracting more flags.

23
As δ grows, the difference between φ r and φ l grows more slowly than with the 24 Relative model. This is because in the Relative model we test larger differences (varying 25 δ between 0.1 and 0.9) than in the Subtraction model (varying δ between 0.025 and  Also in this case, we see that for larger and larger δ differences, the right peak tends 34 to move towards zero. Note that in the Relative model high δ means little difference, Fig S1. The flag distributions in the Relative model for varying levels of tolerance φ r and fixing φ l = δφ r , with δ varying from 0.1 (light blue) to 0.9 (dark blue). The plot reports the probability that a flag (y axis) will be assigned to a source with a given polarity (x axis).
gradients in Figures S1 and S2 go in opposite directions. We now perform an additional 37 analysis to properly interpret this observation.
38 Figure S3 shows the average source polarity once we perform the gradient descent by 39 using the flag distributions we see in Figures S1 and S2. Figure S3 Figure 5 in the main paper: rather than showing the full distributions 42 as we do in the main paper, here we only show the mean of the distribution. 43 We can see that, in both models, for most values of δ the left users are able to shift 44 toward the left (negative) the average polarity of the sources. The only exception is 45 when we have a large difference (i.e. φ l is much smaller than φ r ) in scenario where φ r is 46 already low (≤ 0.3) to begin with. This confirms one of the takeaway of the paper: 47 there is a non-zero bottom for intolerance. When the system reaches a low tolerance 48 value, being less tolerant than this threshold is counterproductive.

49
Also note how there is a sweet spot for δ in the Relative model that follows φ r . For 50 instance, the best value for φ r = 0.8 is δ = 0.6. Values either higher of lower than 0.6 51 for δ will result in a weaker attraction of sources. This confirms that the trivial 52 interpretation of our results ("the lowest tolerance the best") is incorrect. It also shows 53 how the optimal flag distribution in Figures S1 and S2 does not have a right peak in the 54 middle, as one might naively expect, but still needs to be decisively on the right. The The plot reports the probability that a flag (y axis) will be assigned to a source with a given polarity (x axis). fact, as noted in the main paper and shown above, it is always possible to find a pair of 60 Relative and Subtraction δ values that would result in the same φ r -φ l pairing.
In the main paper, we initialize the models by using a realistic distribution of polarity 63 following homophily, and a realistic shape of the social and audience networks. In this 64 section we test what happens when polarity and social/audience connections are 65 distributed randomly. Just like in the main paper, we perform 30 independent 66 initializations and we report the aggregated results.

67
In this Relative Random model, each user assumes a value from the polarity 68 distribution that is independent from the ones of its neighbors. The social network is an 69 Erdos-Renyi random graph with the same number of nodes and roughly the same 70 number of edges as the original network. The audience network is the same, with the 71 additional constraint of being a bipartite user-news source network. We ensure that the 72 networks are connected in a single connected component.   In the main paper we argue that there is a correlation between the bias of a news source 86 and its trustworthiness -i.e. neutral sources are more trustworthy. We also argue that 87 most news sources are trustworthy. Here we support these claims by analyzing data 88 from https://mediabiasfactcheck.com/ a website aggregating fact-checking 89 information that has been used in multiple literature studies [1][2][3][4][5][6].

90
The website contains information about thousands of news media websites and uses 91 two classifications: their political bias (left, neutral, right) and their level of factual 92 reporting (high, mixed, questionable). We count all the sources that have both pieces of 93 information reported. First, the plurality of sources (46%) have high factual reporting. 94 This is in line with our initialization of the model, where 43% of sources have a t s score 95 higher than 0.85, which indicates high factual reporting. 96 We support our claim of correlation between bias and trustworthiness by showing, in 97 Figure S4, that the likelihood of being trustworthy is much higher for neutral sources 98 than for sources with any bias. This is still true if we ignore the left-right distinction: